Course Recommendation Engine
User Interviews + Prototype Testing
Overview
IDP is a global leader in international student services, with a huge network spanning 100 offices across 30 countries. IDP's study abroad app is built to serve all student needs such as providing course and university information, study advise, application assistance, visa processing etc.
Course recommendation engine is AI driven and forms the core of the app which enables students to find the best matching courses and suggestions based on their interests and credentials.
Research Objectives
Part 1: To understand
- How students search and choose their desired course to study abroad (criteria, non-negotiables)
- What are their challenges and pain points
Part 2: To evaluate and identify
- The subjective accuracy of recommendations in the prototype
- If users are comfortable with the number of recommendations
- If there's significant difference between search results and recommendations
- Most likely user actions (view details, shortlist, remove)
- If users are comfortable with the amount of input and which ones are important
Approach
- Interviews with students to understand their decision making process and challenges in choosing a course
- Moderated usability testing with students to evaluate if the prototype fulfills their needs
My Role
- Managed the research ops (recruitment, scheduling, incentives)
- Created the discussion guide in collaboration with the product owner
- Created a template for note takers to document data
- Moderated the interviews and test sessions with students
- Collected and synthesized data to derive themes
- Uncovered insights and presented a report with recommendations
- Created a retrospection document to identify process gaps
Duration: 8 weeks
Tools Used: Hotjar, Google forms, Calendly, Zoom, MS office suite
Outcome
Part 1: The interviews helped us to uncover valuable insights such as
- University is the top criteria for some students, and they are flexible with the course
- It’s challenging to find niche courses online hence they approach consultancies, but they aren’t helpful either
Part 2: The prototype testing revealed that
- The course recommendations provided are not very accurate
- The number of recommendations is high and infinite scroll is not preferred
- Shortlisting is not a common action, users have other ways (diary, excel sheets) to bookmark courses
- Users are willing to give more input to get more refined results
Impact
- A tinder like swipe feature was implemented in the onboarding process to engage users and provide relevant recommendations
- Major changes were made to the tool algorithm which significantly enhanced the accuracy of course recommendations
- A variant of the tool was built for internal counsellors to improve student placement services
Artefacts
